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ABSTRACT 


TEAMS Distance Learning is an Educational 



Telecommunications Network service of the Los Angeles County Office 
of Education (California) . It provides live, interactive 
instructional telecasts for students in grades two through six and 
their teachers and parents across the United States and in Canada. 
This evaluation summary consists of an overview of the evaluation 
design and products of the evaluation. Thirty-seven schools which 
used TEAMS were selected as Planning and Evaluation (PEP) sites from 
regions across the country. Information and data were collected 
through survey instruments and PEP site visits. A TEAMS Student 
Progress Form was sent to every TEAMS teacher as part of the survey. 
TEAMS focuses on a Three-Tier Distance Learning Staff Development 
Model for teachers that includes theoretical training, implementation 
training, and simultaneous teacher training and student instruction. 
There are seven Levels of Use identified in the Concerns Based 
Adoption Model, and staff who are adopting an innovation will move up 
these levels. Teachers reported that they viewed the TEAMS television 
teacher as a role model and demonstrator of new teaching methods and 
as a result, they were able to move easily through the levels of 
adoption. Based on the results of the 1992-96 TEAMS evaluation, an 
Implementation Model for TEAMS was validated. Teachers, students, 
principals, and site coordinators reported that they liked TEAMS 
programming and that it was increasing time allocated to and 
teaching/ 1 earning skills in mathematics and science. Nine tables 
present results of student progress attributed to TEAMS, changes in 
the Distance Learning Professional Development Model, and factors 
that facilitated or impeded the use of TEAMS. (AEF'^ 



/r* /r /r * /r * * * * * * * 'k k * Vc Vc kkkkk k k k k k k k k k k k k k k k k * * * kkkkkkkk k k k k k kkkk k k k k k k k 

* Reproductions supplied by EDRS are the best that can be made * 

^ from the original document. * 

kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk 



394 505 




U S DEPARTMENT OF EDUCATION 

O" ct' o* f djcalionai Reicarcfi inO improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

□ Tr.s vioournem h.is been reproduced as 
HH e'vOd *rc n the person f>r organi/ahon 

j'igtiidhny it 

□ Minij' ctianges have beet' made to 
•mpn've repioduc quality 

• Pointc; ni vte* or opio.ons stated .n this 
dOLui'ieot do not nei essanly represent 
otfiCia! OERl position ot policy 
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TEAMS Distance Learning is an Educational Telecommunications Network (ETN®) service of the Los Angeles 
County Office of Education (LACOE). The TEAMS Project was funded through the Star Schools Programs of the 
United States Department of Education, Office of Educational Research and improvement (OERl) from 1990-96. 

TEAMS provides live, interactive instructional telecasts for students in grades two through six, their teachers 
and parents across the country and in Canada. Programs are designed by specialists in curriculum, instruction and 
parent education, using input from students, parents and staffs of participating agencies. Programs are produced 
and telecast through a cooperative effort betvi/een the TEAMS staff and ETN® 

Evaluation Plan 

The 1 992-96 evaluation plan for the TEAMS Star Schools Program was designed as a formative and summative 
data collection. It provides data from the project as a whole, as well as in-depth data from designated evaluation 
sites across the country. The research study focused on answering questions about: 

• Impact of TEAMS on students, parents and staff 

• Adoption and institutionalization of TEAMS in each area, including its impact on systemic reform 

• How TEAMS promoted the adoption of the National Educational Goals 
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Evaluation Design 

The TEAMS evaluation design was based on the Context. Input, Process, Product (CIPP) Evaluation Model 
developed by Daniel Stufflebeam, et. al. It also contains the major elements of Concerns Based Adoption Model 
(CBAM) which measures the adoption of an innovation. It was developed by Gene Hall and Susan Loucks. 

Part I: Overview of Evaluation Design 

A. Project Goals and Objectives 

B. Overall Project 

1 . Context: How is the TEAMS project organized? 

How is each partner region organized for TEAMS? 

How has TEAMS developed in that region? 

2. Input What resources has TEAMS provided In each region? 

What resources were added through the PBS pilot with PMN? 

What resources were added through other partnerships? 

What resources have states, regional agencies, districts, schools and others provided? 

3 Process 

a. Installation: 



b Implementation: 



How have districts, schools, teachers been selected to participate? 

What are patterns of beginning implementation of TEAMS? 

How have TEAMS programs been delivered? 

What technical assistance has been given? 

What support materials and process are available? 

Teacher Involvement 

How are first through sixth year users involved with TEAMS? 

4. Product (Outcomes) 

How many participants, districts, states received services; what kinds of services were received? 

What are the participants' demographic characteristics? 

What is the difference in using live or tape versions of programs? 

What types of interaction creates the greatest benefits? 

What have been the benefits to teachers, students, parents and administrators? 

What are the effects of being part of a national telecommunications project? 

What are the effects of being part of TEAMSNet? 

What outcomes resulted from the PBS pilot with PMN? 

What outcomes resulted from other partnership? 

In-depth data from selected TEAMS sites 
1 Context School data 

Support structures 
Resources 
Reform efforts 
Demographics 

Testing, assessment information 
Behavioral and attendance information 
TEAMS student data 

Demographics permission to reproduce this 

Testing, assessment Information material has been granted by 

Behavioral and attendance information 
TEAMS Teacher data 
Demographics 
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Education and training 
Teaching experience 
Participation in reform efforts 

2 Input TEAMS Planning; Resources Used for TEAMS 

3. Process Installation: How did the school learn about TEAMS? 

How did they decide to use TEAMS? 

How did they get ready to use TEAMS ? 

Organization: How are they organized to use TEAMS ? 

Implementation . How do they use the TEAMS programs? 

How do the teachers prepare tor the TEAMS programs'^ 

How do the teachers facilitate and manage the TEAMS programs? 

How do they use the support materials? 

What pre and post activities do they use? 

How do they assess the Impact of the i EaMS programs? 

Program Series Evaluation: How do they rate the TEAMS program series? 

CBAM: What Stages of Concern are teachers In at various points of implementation; 

Beginning use of TEAMS: Firet through sixth year of TEAMS use 
What level of use is related to each year In TEAMS? 

What is the innovation configuration of TEAMS at the site 

4. Product: How has TEAMS benefited the school as a whole? 

What changes have students experienced from TEAMS? 

What benefits have teachers experienced from TEAMS? 

What benefits do parents see or experience? 

Part il: Products of the Evaluation 

A Report on Organization, Installation, Implementation and Impact of TEAMS 

B. TEAMS School Intervention Plan 

C. Teacher Involvement and Use of TEAMS by Year in Program 

D. Successful TEAMS School Site Models 

Planning and Evaluation Project (PEP) Sites 

Each school which used TEAMS was asked to nominate itself as a Planning and Evaluation Project (PEP) Site. 
Regional TEAMS Coordinators worked with schools to complete the nomination forms. Nomination forms required 
data on demographics of the school and agreement to in-depth study techniques including the site visit, if they 
received the nomination. Each region nominated several more schools than were required so that the evaluation 
team could make the final PEP school selection based on demographics that reflected diversity. Originally 35 sites 
were to be selected, but two others were added (one in Washington, DC and one in Utah) bringing the total to 37 in- 
depth study sites. In 94-6, new partners provided sites for implementation site visits. 

Methods 

Information and data were collected for the evaluation through t\wo primary methods, survey Instruments and 
PEP site visits. Survey instruments were developed for administrators at the PEP in-depth study sites, teachers at 
PEP in-depth study sites, TEAMS teachers at regular sites, TEAMS regional coordinators, and Pacific Mountain 
Network (PMN) public television station affiliates. 

Site visits were made to the PEP School sites during each year. At each site, focus interviews were conducted 
with principals, teachers, resource teachers, other Involved staff and, at some sites, with the students. Classroom 
use of a TEAMS program was observed where possible. TEAMS programs were not always being broadcast or used 
on the day of the site visit. Most site visits were conducted in the classroom where the students used TEAMS as 
substitute teachers were not available. 

Pep sites were located In Los Angeles County, Los Angeles, Boston, Detroit, St. Louis, Kansas City, 
Washington, DC, Utah, Arizona, Missouri, North Carolina, Georgia and New Mexico., 

Re spondents 



Respondents 


1993 


1994 


1995 


PEP School Administrators 


73 


37 


43 


PEP Site TEAMS Teachers 




113 


125 


TEAMS Teachers 


203 


330 


335 


TEAMS Regional Coordinators 


8 


8 


11 


Pacific Mountain Network Af'liates 




9 


10 


Students 


284 


7,299 


8,235 



Summary of Results 

A TEAMS Student Progress Form was sent lo every TEAMS teacher as part of the survey instrument. The form 
collected basic information on (gender and participation In Chapter 1, LEP, Gifted, and Special Education programs. 
The form asked the TEAMS teacher to describe the degree of the outcome for each student that could be attributed 
solely to using TEAMS. The scale of one to four was used where four was a great degree and one was none. 
Teachers ranked the 7,299 in 1994 and 8,235 in 1995 students on the following attributes. 
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improved Content Knowledge and Skills 
Improved Critical Thinking and Problem Solving 
Improved Language Skills 
Increased Interest in the Subject Area 
Improved Quality of Work 
Increased Interest in School 
Improved Attendance 
Improved Behavior 

Takes Responsibility for Own Learning 
Greater Confidence as Learner 
Higher Self-Regard 

Usable data was returned for 7,299 students in 1994, and 8,235 studsents in 1995, make this the largest study 
conducted with students participating in a Star Schools Project. It is also the largest study conducted of students in 
a distance learning program during this period of use of technology for education. While other studies of student 
learning have generally focused on whether students learn as much or as well from educational technology as from 
traditional classrooms, the TEAMS evaluation focused on students using the same programs and the same 
technology, but studied other educational outcomes as an indication of the success of the TEAMS program. 

The evaluation also focused on new ways to gather information about student learning outcomes. Wtiile it has 
become apparent that students learn as well from educational uses of technology, the use of standardized test 
scores does not report all of the learning that Is taking place. It is important that the developers of standardized 
tests begin to Include ways to measure the new skills that students are learning, that society va‘ues, and are 
mandated through the Educate America Act: Goals 2000.. 

The TEAMS student progress form created for the TEAMS Star Schools Project endeavored to develop a 
significant new method to collect and assess student learning and improvement through distance learning programs. 
The form was successfully used by teachers and is used throughout the evaluation of TEAMS Star Schools Project. 

Results of Student Progress Attributed to TEAMS 

A series of multiple linear regressions were performed on the data. For all of the student results, no noted 
change In a variable did not mean that the student was working at an F grade level. TEAMS programming Is seldom 
used as the only source of mathematics or science education in the classroom. Teachers may have been unable to 
see a change which was directly attributable to TEAMS programming and thus recorded a score of one (no change 
due to TEAMS) rather than a score of two to four showing change due to TEAMS. For example, many tea ' irs noted 



on their survey Instruments that students always came to class and there was little room for improvement in the 
question about Increased attendance. Other teachers noted that the behavior in their classrooms was always good 
and thus, a score of one for no change, was not to be a considered a bad mark or a poor reflection on TEAMS. Simple 
regressions were conducted using “improved content knowledge and skills" as the dependent variable. All 
independent variables were significant at the level of P=.0001 (see Table 1). 

Table 1: Simple Regressions on the Dependent Variable 

Improved Content Knowledge and Skills 

Independent Coefficient Std Std. t- P- 

Vanable Error Coeff. Value Value 


Intercept 




2.944 


.013 


2.944 


223.680 


<0001 


M/F 




-.038 


.019 


-.024 


-2 025 


.0429 


Intercept 




3.010 


.013 


3.010 


233.948 


<0001 


Ch 1 




-.179 


.019 


-111 


-9.564 


<0001 


Intercept 




2.976 


Oil 


2.976 


271.799 


<0001 


LEP- 




.189 


.021 


-.104 


-8.915 


<0001 


Intercept 




2.895 


9.810E-3 


2.895 


295.069 


<0001 


Gifted 




.332 


.032 


.120 


10.303 


<0001 


Intercept 




2.950 


9.700E-3 


2.950 


304.092 


<0001 


Soecial Ed 




-.314 


.033 


-.112 


-9.639 


<.0001 


Intercept 




.625 


.019 


.625 


32.476 


<0001 


Thinkina 




.807 


6.481 E-3 


.824 


124.440 


<.0001 


Intercept 




1.504 


.024 


i 504 


63.633 


<.0001 


LanQuaqe 




.541 


8.514E-3 


597 


63.493 


<0001 


Intercept 




.942 


.025 


.942 


38.164 


<.0001 


Subiect Interest 




.659 


7.901 E-3 


.699 


83.449 


<.0001 


Intercept 




1.432 


.024 


1.432 


60.148 


<.0001 


Quality 




.563 


8.534l:-3 


.611 


65.992 


<.0001 


Intercept 




1.629 


.024 


1.629 


67.763 


<0001 


School 




.490 


8.587E-3 


.555 


57.036 


<0001 


Intercept 




2.352 


.021 


2352 


112.473 


<.0001 


Attendance 




.269 


8.889E-3 


.334 


30.246 


<0001 


Intercept 




2 183 


022 


2 183 


100.669 


<0001 
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330 8.847E-3 .400 37.330 <0001 



Intercept 


1.645 


.023 


1.645 


70.448 


<0001 


ResDonsibilitv 


.494 


8.483E-3 


,563 


58.194 


<0001 


Intercept 


1.502 


.024 


1.502 


62.995 


<0001 


Confidence 


.531 


8.424E-3 


,594 


62.992 


<0001 


Intercept 


1.596 


.024 


1.596 


67.678 


<0001 


Reaard 


.503 


8.441 E-3 


.572 


59.612 


<0001 



In the multiple linear regression which used “improved content knowledge and skills” as the dependent variable, 
ten independent variables were found to be significant at P= .05 and above. Improvement was significant for Chapter 
1. LEP and Special Ed TEAMS students. Teachers throughout the US. saw improvement for these students in 
Improved critical thinking and problem solving, language skills, increased interest in the subject area, improved 
attendance and behavior, and Improvement in the responsibility the students took for their own learning as well as a 
positive increase in student self regard (see Table 2). 



Table 2: Results of Multiple Linear Regression on Content Knowledge and Skills 



Variable 


P 


Chapter 1 


P=.0005 


LEP 


P=.0072 


Special Ed 


P=.0113 


Improved Critical Thinking and Problem Solving 


P=.0001 


Improved Language Skills 


P=.0001 


Increased Interest in the Subject Area 


P= 0001 


Improved Attendance 


P=.0001 


Improved Etehavlor 


P=.0324 


Takes Responsibility for Own Learning 


P=.0333 


Hioher Self-Regard 


P=.0019 



A multiple Irnear regression was done using “increased interest in the subject area" as the dependent variable. 
Nine independent variables were significant in this procedure. TEAMS teachers saw improvement for special 
education and gifted students In improved critical thinking and problem solving, quality of work, increased interest in 
school, improved attendance and behavior, and improvement in the confidence students had in themselves as 
learners (see Table 3). 



Table 3: Results of Multiple Linear Regression on Increased Interest in Subject Area 



Variable 


P 


Gifted 


P=.0001 


Special Ed 


P=.0206 


Improved Content Knowledge and Skills 


P=.0001 


Improved Critical Thinking and Problem Solving 


P=.0001 


Improved Quality of Work 


P=.0001 


Increased Interest in School 


P=.0001 


Improved Attendance 


p=.0001 


Improved Behavior 


P=.0177 


Greater Confidence as Learner 


P=.0001 



To determine specifically what improvements teachers were attributing to the four groups of students, multiple 
linear regressions were performed for each of the four groups individually (as the dependent variable) and using the 
twelve assessment criteria as Independent variables. 

For Chapter 1 students, the independent variables that were significant were Improved content knowledge and 
skills. Improved critical thinking and problem solving, improved language skills, improved quality of work, improved 
attendance, and a higher self-regard (see Table 4). 

For gifted students, the Independent variables that were significant were gender, improved critical thinking and 
problem solving, improved language skills, Increased interest in the subject area, improved quality of work, increased 
interest in school, improved attendance, Improved behavior and a higher self-regard (see Table 5). 

For limited English proficient (LEP) students, the independent variables that were significant were improved 
content knowledge and skills. Improved critical thinking and problem solving, improved language skills, increased 
interest in the subject area, improved quality of work, improved attendance, an increase in taking responsibility for 
their own learning, and a higher self-regard (see Table 6). 

For special education students, the independent variables that were significant were gender, improved content 
knowledge and skills, improved critical thinking and problem solving, an increase In taking responsibility for their own 
learning, and a greater confidence as a learner (see Table 7). 

The Intercept P on each of these groups of students shows that the TEAMS (Apollo 2000) program is 
significantly impacting the learning of these students 
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Table 4: Results of Multiple Linear Regression on Chapter 1 Students 
v^hapter 1 Students Intercept P=.00( 



Variables 


P 


Improved Content Knowledge and Skills 


P=.0001 


Improved Critical Thinking and Problem Solving 


P=:.0001 


Improved Language Skills 


P=.0001 


Improved Quality of Work 


P=.0004 


Improved Attendance 


P=.0403 


Higher Self-Regard 


P=.0001 



Table 5: Results of Multiple Linear Regression on Gifted Students 
Gift ed Students Intercept P= 0525 



Variables 


P 


Maleypemale 


P=.0724 


Improved Critical Thinking and Problem Solving 


P=.0001 


Improved Language Skills 


P=.0255 


Increased Interest In the Subject Area 


P=.0001 


Improved Quality of Work 


P=.0001 


Increased Interest in School 


II 

b 

o 

o 


Improved Attendance 


P=.0025 


Improved Behavior 


P=.0463 


Higher Self-Regard 


P=.0031 



Table 6: Results of Multiple Linear Regression on LEP Students 



Variables 


P 


Improved Content Knowledge and Skills 


P=.0001 


Improved Critical Thinking and Problem Solving 


P=.0004 


•niproveo Language Skills 


p=.booi 


Improved Quality of Wor1< 


P=.0019 


Improved Attendance 


P=.0023 


Takes Responsibility for Own Learning 


P=.0072 


Higher Self-Regard 


P=.0079 



Table 7: Results of Multiple Linear Regression on Special Education Students 
Special Education Students Intercept P=.0001 



Variables 


P 


Male/Female 


II 

b 

o 

o 


Improved Content Knowledge and Skills 


P=.0146 


Improved Critical Thinking and Problem Solving 


P=.0035 


Increased Interest In the Subject Area 


P=.0156 


Takes Responsibility for Own Learning 


P=.0012 


Greater Confidence as Learner 


P=.0006 



TEAMS Distance Learning Staff Development Model 

TEAMS focuses on a Three-Tier Distance Learning Staff Development Model for teachers (Cassidy, 1990). The 
approach includes: 

1 . Theoretical Training: information, theory, demonstration and two-way communication about the theoretical basis 
of the TEAMS Instruction and training 

2. Implementation Training: theory, demonstration, practice and peer discussion of curriculum and Instructional 
methods involved In the student programming, providing training to implement the student programs. 

3. Simultaneous Teacher Training and Student Instruction: teacher training through in-class eyoerience, practice 
and support from the studio team-teacher, through live, interactive student instructional programs. For TEAMS, 
this three-tiered approach has provided answers for many problems associated with traditional staff 
development design. 

• It Is long term, sequential training 

• It fosters immediate transfer of learning, with new skills bcx^oming a part of the teacher's repertoire of 
instructional methods 

• It is primarily conducted in the teacher's classroom during the regular school day 

• It creates Immediate changes In the roles of the teacher and student 

• It provides opportunities for teachers to see students being successful with a rich and challenging curriculum. 
This allows them to change attitudes and behaviors related to Instruction and expectations of their students. 

• It provides motivation for teachers to participate in other staff development after the regular school day 
because it Is directly related to their classroom program 



O 
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This model, developed by Sheila Cassidy for TEAMS Distance Learning, is based on research and practice in 
the fields of staff development and adult learning, as well as national and state standards and guidelines. The basis 
of the staff development research is formed by work by; Joyce and Showers; Cassidy and Taira; and the Rand 
Corporation. The adult learning principles are summarized in work by Jones and Woodcock. 

The staff development research (Joyce & Showers, 1988) provides compelling data on the relationship between 
training outcomes and specific training components. They analyzed the relationship between the training outcomes 
of knowledge, skill and transfer of training for participants engaged in programs options providing . 

1 . Information 

2. Theory 

3. Demonstration 

4. Theory and demonstration 

5. Theory and practice 

6. Theory, demonstration and practice 

7. Theory, demonstration, practice and feedback 

8. Theory, demonstration, practice, feedback and coaching 

Their research clearly shows that training which provides only information and theory produces only increased 
knowledge in participants. That encompassing any of options numbers four through eight shows greater knowledge 
and skill outcomes. Option eight provides the greatest outcomes in knowledge, skills, and transfer of training. 
Practice, feedback, and coaching can be considered an in-classroom, on the job, experiential and support 
component. With its three tiers, TEAMS provides a distance learning alternative to option eight. It clearly provides 
theory, de .onstration and practice. Although distance learning cannot provide a full face-to-face feedback and 
coaching component, part of what feedback and coaching provides Is an in-class support system. That is provided 
through the In-class team teaching with the studio instructor. 

In retraining of teachers, Cassidy and Taira (1988, 1989) found that teachers reported the factors which 
contributed to their success were: a sound theoretical basis; experience anJ practice with the particular curriculum 
and instruction being adopted/adapted; a support system designed specifically to their needs; convenience, with 
training during the school day and at their own site when possible; training with no expense to the teacher. The 
simultaneous in-class training component of TEAMS meets ail of these criteria. 

The Rand Corporation found that successful projects had these common characteristics for staff development 
(Berman and McLaughlin, 1978): 

1 . Training is concrete, continual, and tied to the world of the teacher 

2. Local resource personnel provide direct follow-up assistance 

3. Peer observation and discussion provide teachers with reinforcement and encouragement 

4. School leader participates in staff development 

5. Regular meeting held with teachers for problem solving and adapting techniques and skills of the innovation 

6. Released time used for teacher staff development 

7. Staff development planned with teachers prior to and during the project 

Cassidy (1985) reviewed programs with findings similar to the Rand study but with additional Information. 

1 . Individualized staff development activities are more effective than large-group activities 

2. Programs using demonstrations, trials, and feedback of ideas are more effective than lecturing and reading. 

3. Staff development programs are more successful when teachers are active planners and help each other. 

Jones and Woodcock (1984) describe these adult learning principles: 

1 . The adult is a partner with the instructor in the learning process 

2. Adults are capable of taking responsibility for their own learning 

3. Adult learners gain through two-way communications 

4. Adults learn through reflection on their and others' experience 

5. Adults learn what they perceive to be useful in their life situations 

6. Adults' attention spans are a function of their interest in the experience 

7. Adults are most receptive to instruction that is clearly related to problems they face daily 

8. Adults learn best when they are treated with respect 

9. Adults do not typically see themselves as learners 

1 0. Adults learn better In a climate that is informal and personal 

1 1 . Adult learners apply learning’s that they have been influential in planning 

1 2. Adults learn when they feel supported in experimenting with new ideas and skills 

1 3. Adults may have somewhat fixed points of view that make them closed to new ways of thinking and behaving 

1 4. Adults learn to react to the differential status of members of the group 

15. Adults are internally motivated to develop increased effectiveness 

1 6. Adults filter their learning through their value system 

Levels of Use 

There are seven Levels of Use identified in the Concerns Based Adoption Model (CBAM), and staff who are 
adopting an Innovation will move up these levels in seven different areas During the two years of this evaluation, 
TEAMS teachers were surveyed and interviewed to determine at what level of use they were working. By determining 
their level of use and the time each takes to move through the levels, ‘ lay be possible to plan an implementation 
strategy that will reduce the time to adopt the innovation of distance le* . ring, and specifically TEAMS programming. 
The levels and areas are as follows. 
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0: Non-use: State in which the user has little or no knowledge of the innovation, no involvement with the 

innovation, and is doing nothing toward becoming involved. 

1 : Orientation: State in which the user has recently acquired or is a xiuiring Information about the Innovation 

and/or has recently explored or is exploring Its value orientation and its domand upon user and user system. 

1 1 : Preparation: State In whlc;h the user Is preparing for first use o' the innovation. 

Ill; Mechanical Use. State In which the user focuses most effort on the short-term, day-to-day use of the 
innovation with little time for reflection. Changes In use are made more to meet user needs than client needs. 

IV A: Routine: Use of the Innovations stabilized. Few If any changes are being made in ongoing use. Little 
prc paratlon or thought Is being given to Improving innovation use or its consequences. 

IV B: Refinement: State In which the user varies the use of the Innovation to increase the impact on clients 
within Immediate sphere of influence. Variations are based on knowledge of both short and long term conseqi-ences 
for clients. 

V: Integration: State In which the user Is combining own efforts to use the innovation with related activities of 
colleagues to achieve a collective Impact on clients within their common sphere of Influence. 

VI: Renewal: State In which the user re-evaluates the quality of use of the innovation, seeks major modifications 
of or alternatives to present Innovation to achieve increased impact on clients, examines new developments in the 
field, and explores new goals for self and the system. 

A New Model of Teacher Training 

The pattern that emerged during the evaluation has created a new model for teacher preservice and In-service. 
TEAMS teachers reported that they viewed the TEAMS television teacher as a role model. As a result, they were able 
to move easily through the levels of adoption. 

First year TEAMS teachers continued to report that there was a great deal of preparation. They read the printed 
materials provided, set out the materials for students, and watched the programs with students. First year TEAMS 
teachers who used the program on videotape usually previewed the tape. Rrst year TEAMS teachers reported that 
they felt that the programs required extensive study by them to learn the new instructional methods. However, they 
felt the time was profitable because their students were learning so much more and enjoyed the new Instructional 
methods. First year TEAMS teachers move through the third level of use. 

Second year TEAMS teachers reported that because they now had an understanding of the Instructional 
methods as well as TEAMS, they spent significantly less time preparing for programs. Their higher level of comfort 
with the methods gave them the confidence to use the methods in other math or science classes with their students. 
These methods included hands on, discovery, and collaborative group learning. Many teachers described the 
television Instructor as a role model. They gained confidence in their skills because the television teacher provided 
step by step guidance in presenting material to students. Teachers reported that they received rrx)re usable 
information on new instructional methods through TEAMS programming than through in-service seminars. Second 
year TEAMS teachers are in the IV A level of use - routine. 

Third year TEAMS teachers continued to report that they were very comfortable with programming and 
instructional methods. They spent a small annount of time gathering the class materials for programs. This group 
also reported that the Instructional methods had become natural extension of their teaching style. They use the new 
instructional methods across the curriculum and appear to have full adopted the methods. Third year TEAMS 
teachers are in the IV B level of use - refinement. 

Fourth year TEAMS teachers reported that they have fully adopted th3 Instructional methods embodied In the 
TEAMS programming. They continue to use the TEAMS programs because students enjoy it and learn from it. "^hey 
continue to use their new methods across the curriculum. These instructors have also become mentor to new 
TEAMS instructors at their schools. Fourth year TEAMS teachers are In the V level of use - integration. Some have 
moved to the VI level of use - renewal. 

Using TEAMS has effectively provided teachers with new methods which they use because they have watched 
the television teacher demonstrate the methods. Immediately after viewing the prograrri, TEAMS teachers apply the 
methods with their students. These results were reported across the United States at all evaluation sites as well as 
in the surveys. Principals also noted these changes in teachers saying that TEAMS teachers showed more 
enthusiasm for math/science, a higher use of interactive and hands-on methods, and that teachers were more 
confident of their ability to teach math and science, (on a scale of one to four where four is high) was 2.4; second 
year TEAMS teachers' mean was 2.5; and third year teachers scored the question at a mean of 2.7. 

TEAMS teachers continued to report an increase in the ability to teach heterogeneous groups, teach 
math/science in an active learning environment, manage a class of students using manipulatives, use cooperative 
learning in mathfecience instruction, involve parents in their child's math/science education, use the textbook as a 
resource rather than as the primary instructional tool, use a variety of alternative assessment strategies, and follow 
national mathematics standards/science recommendations. Tlie rrwdel that has evolved from TEAMS is many 
faceted. A comparison of the existing professional development rx\o6e\ and the new TEAMS distance learning 
professional development nnodel is shown in Table 8. 
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Table 8: Distance Learning Professionai Development Model 



Existing Professional Dev6!^pment Model 


New DL Professional Development Model 


Face -to-face 


Distance learning delivered 


Inservice day (2-4 per year): 8-24 per year 


Weekly 1 -1 /2-2 hours per week: 64 hours per year 


Travel to inservice site 


Delivered to teacher's classroom^ite 


In-service Instructor has a limited ability to develop 
as a role mode! for the teacher 


Role model provided to teacher by distance learning 
instructor 


Large inservIce group 


One-to-one 


Short demonstration 


Full step-by-step and frequent demonstration 


Limited examples 


Variety of examples 


Very limited hands-on 


T wice weekly hands-on 


Support: Limited access to follow-up with in. jrvice 
provider 


Support: Direct and frequent access to distance learning 
instructor via television, telephone, fax, computer 


Limited opportunity to process infonnation/apply it 


Provides opportunity to process Information/apply it 


Limited application of new information 


Immediate application of new information 


Students are seldom included in in-service 


Provides opportunities for teachers to see students being 
successful; allows attitude and behavior changes related 
to instruction and expectations of their students. 


Print materials limited to handouts 


Full print materials which provide theory, information, 
methods, and implementation for student programs 


instructional materials are not provided 


All instructional materials are provided 


Use text as the primary instructional tool 


Used the text only as a resource 


General instructions 


Content specific instructions 


No opportunity to review 


Tape review 


Costs: non-teaching days or substitute 


Costs: nothing additional 

Two for one - students and teachers 


Instruction only for the teacher 


Simultaneous teacher training and student instruction: 


Results: Limited 


Results: Significant change 


Little change in teaching methods 


Significant change in teaching methods 


Limited increase in content teaching time 


Increase in content teaching time 


Limited gain in non-specialty content area 


Significant gain in non-specialty content area knowledge 
and comfort level 


Limited gain confidence to teach non-specialty areas 


Significant gain confidence to teach non-specialty areas 


Limited increase in use of instructional methods 
across the curriculum 


Uses new instructional methods across the curriculum 


Limited increase in mentoring 


Become mentors to new instructors 


Limited movement to higher levels of use 


Moves teachers to higher levels of use 



TEAMS Distance Learning Program Implementation Model 

Based upon the results of the 1992-1996 TEAMS evaluation, an Implementation Model for TEAMS was validated. 
There are a variety of elements to the model which include components for the site implementation and components 
for the instructional program series to be fulfilled by the TEAMS staff. 

For a large distance learning project such as TEAMS, an Implementation model is necessary so that the adoption 
of the innovation of distance learning is successful at each site. As more districts subscribe to the program, it is 
important that they have a plan to follow which clearly defines the steps to follow to ensure a successful adoption. In 
the early days of the TEAMS program, staff members could spend rrx>re time working with districts and sites. As the 
number of sites and partners increased. It became impossible for staff to spend the same amount of time with new 
receive sites. It has also become a problem for TEAMS regional coordinators to spend as much times with sites 
because the program use has expanded significantly in the areas of each city or state partner. Because of this, an 
implementation model is necessary. It provides a best practices nrx)del for all adopters of the TEAMS program. 

The Implementation rrx)del will be helpful for TEAMS staff, regional coordinators and for site coordinators as a 
procedure to follow in helping new users successfully adopt TEAMS. As a recommended set of procedures to follow 
at a sits, it provides the Impetus for the site coordinator to follow the suggestions, or to emphasize the necessity of 
following the recommendations to administrators and teachers to ensure a successful adoption. 

The implementation model is also useful to analyze a TEAMS site that has not had a successful adoption of 
TEAMS. By using the guidelines as a checklist, it could be determined how many of the recommended procedures 
were not followed. If very few guidelines had been followed, the site would provided with a reason for an 
unsuccessful implementation and adoption of TEAMS. Then, the guidelines could be followed and those steps taken 
if it thought that the low adoption could be reversed. 

The model Is useful as an lmplemp''tatlon guideline for all distance learning programs with modifications for 
certain program components which may or may not be a component of TEAMS. The component of the rrodel include 
how the program is introduced, institutionalization of the program, technical dimensions, and overall program design 
As more schools use TEAMS, it is Impossible for staff to visit every school and assist firsthand with the 
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Implementation. As a result, it becomes important to have a clearly defined statement of the implementation plan that 
has been most successful at TEAMS sites throughout the United States. The validated implementation plan can be 
used by TEAMS staff to determine If the current implementation recommendations have been useful and to adjust 
and add new implementation methods that have been validated. This will ensure that the TEAMS program is 
effectively implemented at state departments of education, state regions, districts, schools and In the classroom. 
The validated implementation rrx)del will also ensure a cost efficient implementation and a productive start-up for new 
TEAMS users. 

Problems 

Very few problems were mentioned by teachers. Program times seldom meshed with schools in any time zones. 
Problems in receiving printed materials which were reported In the first year of the evaluation, diminished in the 
second. Few technical problems were reported, but more interest in the use of educational technologies resulted in 
requests for more funding to purchase computers, program kits, telephone lines, and additional television sets. Lack 
of funding was consistently mentioned by respondents as a concern as it prevented access for ail students. 

Several districts have moved to a full year model for classes. In the case of one large district (Los Angeles 
Unified School District), students are released for up to two months during the traditional school year. For those 
students who are out of school during TEAMS broadcasts, the use of video tapes is the only option. During the 
summer, these students do not have access to other students, to the distance learning instructor, or other means of 
Interaction. If more schools adopt the full year model, TEAMS should consider programming for the summer months. 



Factors that facilitated the use of TEAMS 


Factors that impeded the use of TEAMS 


Planning and preparation time 


No planning and preparation time 


Kits had all materials including all hands-on materials 


Sharing equipment and kits 


Equipment telephone, VCR, cabled classroom 


Lack of equipment telephone, VCR, cabled classroom 


Cable/Satellite access to program in regular classroom 


Moving to resource classrooms 
viewing programs with other classes 


Printed materials that provided information for teachers 
and work materials for students 


Low photocopy budgets 


Good lesson plans 




District and administrative support 


Low district or administrative support 
Low staff morale 


Ability to tape/show programs when convenient 
professional development programs 


Remembering to tape the program 


Broadcast times; ability to tape programs 


Broadcast times: taping programs 


Bilingual aides 


Lack of bilingual materials 



Summary 

TEAMS was chosen by districts, principals, and teachers for a variety of reasons including the fact that It was 
based on the mathematics and science standads/recommendations, hands on procedurec, and distance delivery, 
which v/ould enhance teaching and learning. Schools and teachers continued to use TEAMS in the second, third, and 
fourth years 'jcause it fulfilled its original promise. 

Students are learning from TEAMS. There are increases in skills in mathematics and science content that 
TEAMS teachers can directly attribute to students participating in TEaMS programming and using TEAMS materials. 
Teachers reported that students who had difficulty learning about mathematics and science through other methods, 
were now learning from the TtEAMS hands-on methods and manipulatives. Students revealed in student focus groups 
that it was fun to learn with TEAMS as opposed to the “other"' way which seemed to be the “hard" way. 

Teachers reported a positive change in student behavior even with normally disruptive students. Teachers 
reported increased self-esteem, increased attendance, and an increased interest by girls in mathematics/sclence. 

Teachers reported that students evidenced their comfort level with new content by using scientific inquiry, 
increased their participation in science fairs, and selecting TEAMS topics for their science fair projects. Teachers 
reported that students are more interested and motivated to do mathematics, including students who were lower 
achievers in math. 

TEAMS has effectively provided teachers with new instructional methods by viewing the TEAMS television 
teacher during the student programming. The nrrcdel has changed the teaching styles and the instructional methods 
of teacher's by the tlrrve teachers have used TEAMS three years. Fourth year TEAMS teachers continue to use 
TEAMS because students continue to benefit from the programs The most significant changes in TEAMS teachers 
were achieved by those who used TEAMS regularly. 

Teachers' ability Increased in a variety of ways. Because of the TEAMS teaching model they reported increased 
skills in teaching heterogeneous groups, teaching mathematics/science in an active learning environment, managing 
the student use of manipulatives, using cooperative learning in iinathematlcs^cience instruction, involving parents in 
their child's mathematics/science education, using the textbook as a resource rather than as the primary 
instructional tool, using a variety of alternative assessment strategies, and following national mathematics 
standards/science recomnrrer^dations 

Teachers, students, principals, and site coordinators reported that they liked TEAMS programming and that it 
was increasing the time allocated to mathematics and science in the classroom. Teachers increased their class time 
in mathematics and science by an average of four hours per week. 
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TEAMS motivates students to learn math and science because they enjoy It and because it maintains their 
enth siasm through interaction with the TEAMS television teachers and the use of hands-on manipulatives for 
learning. TEAMS is also used as a taped program and the student learning in these classes is equivalent to that of 
the students who view the program live. The TEAMS evaluation showed that special groups of students benefit from 
participating in TEAMS programs. These include Chapter 1 , LEP, special educatio'^ and gifted students. 

The Pacific Mountain Network (PMN) pilot program where TEAMS was delivered by public television stations had 
a limited success. Approximately fifty television stations requested information about TEAMS. More promotion, 
awareness, and utilization was undertaken by PMN during 1994 96. 

Based upon the evaluation, the TEAMS Distance Learning Implementation Model was validated. There are a 
variety of elements to the mode) which include components for the site implementation and components tor the 
instructional program series to be fulfilled by the TEAMS staff. The evaluation identified how the TEAMS program 
can be most successfully adopted and Implemented by a district and its schools As now users are continually being 
added, the staff is unable to visit each new school and classrcorr^ ;vhere the program will be used. The TEAMS 
Implementation Model provides a guideline for the staff to review current implementation recommendations and 
emphasize those which are the most useful. This will lead to a cost efficient and smooth startup for new users. 



For More Information. 
Carla Lane, Ed.D. 



WestEd 

510- 551-7405 or 415-241-2726 




TEAMS Star Schools Evaluation 1992-1996 Summary Report 



11 



10 



